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FOREWORD 


This  report  describes  the  development  and  use  of  a  rating  scale  to  measure  F-4  crew 
performance  differences  under  combat  conditions.  The  research  was  tarried  out  under 
Project  6323,  Personnel  Management  Research  and  Development;  Task  632305, 
Development  of  Statistical  and  Mathematical  Procedures  to  Facilitate  Personnel  Research. 
The  findings  of  this  report  were  made  in  partial  response  to  a  Requirement  for  Personnel 
Research,  RPR  68-16,  originated  by  APGC  (PGQT). 

This  report  has  been  reviewed  and  is  approved. 

John  G.  Dailey,  Colonel,  USAF 
Commander 


ABSTRACT 


A  proficiency  rating  form  was  administered  to  F4  crewmembers  following  each  of 
a  series  of  combat  missions.  The  rating  form  was  one  of  several  data  collection 
instruments  developed  as  part  of  Project  Combat  Team  to  gather  behavioral  data  designed 
to  relate  the  differences  in  the  training  of  pilots  and  navigators  to  operational 
performance.  The  rating  categories  were  designed  to  measure  proficiency  in  ten 
second-seater  functions  and  three  general  characteristics  related  to  second-seater 
proficiency.  Using  an  11-point  rating  scale,  aircraft  commanders  compared  their 
second-seat  crewmember  with  second-seaters  of  equal  combat  experience.  Differences 
between  pilot  and  navigator  second-seaters  on  each  rating  category  were  tested  by  a 
multiple  linear  regression  analysis  Data  from  other  instruments  relevant  to  evaluation  of 
mission  success  were  also  presented.  Several  conclusions  were  drawn  on  the  basis  of  these 
data:  (a)  it  is  possible  to  obtain  reliable,  valid  proficiency  data  in  a  combat  environment. 
(6)  Proficiency  ratings  can  be  used  to  measure  the  performance  differences  between  two 
groups  in  an  operational  setting,  (c)  By  the  30th  mission  neither  group  of  second-seaters 
demonstrated  a  general  superiority  in  performance  over  the  other.  (Differences  between 
pilots  and  navigators  early  in  their  combat  experience  were  sharply  reduced  by  mission 
segment  26  to  30,  except  in  performance  of  the  task  Understanding  Radio 
Communications.)  ( d)  The  early  proficiency  differences  between  pilot  and  navigator 
second-seaters  appeared  to  be  a  function  of  training  differences.  Therefore,  it  may  be 
possible  to  reduce  or  eliminate  these  differences  through  modifications  in  crew  training 
programs,  (e)  During  the  first  30  combat  missions,  pilots  flew  on  more  missions  rated  as 
completely  successful  than  did  navigators.  Although  the  differences  were  small,  they  were 
statistically  significant. 


SUMMARY 


Shore,  C.W.,  Cfissn,  C.R.,  RstSf?,  Fit.,  aad  Chiorini,  JJL  Proficiency  differences  of  pilot  end  navigator 

F-4  second-seat  crewmembers:  A  Southeast  Asia  evaluation.  A FHRL-T R-? 0-9 .  Lackland  AFB,  Tex.. 

Personnel  Research  Division,  Air  Force  Human  Resources  Laboratory,  April  1970. 

Problem 

At  the  request  of  the  Department  of  Defense,  the  Air  Force  initiated  a  study  to  determine  the 
feasibility  of  replacing  pilots  with  navigators  as  second-seat  crewmembers  in  F4  aircraft.  The  Department 
of  Defense  estimated  the  change  could  rave  the  Air  Force  $400  million  in  training  costs  over  a  5-year 
period.  The  Air  Force  felt  that  the  change  in  F-4  crew  composition  would  be  feasible  only  if  it  could  be 
demonstrated  that  the  use  of  navigators  in  the  second  seat  would  not  have  a  detrimental  Impact  on  combat 
effectiveness  or  aircraft  safety.  Prefect  Combat  Team  was  established  to  develop  and  conduct  a  series  of 
tests  to  determine  if  there  were  significant  differences  in  the  combat  performance  of  navigator  and  pilot  F-4 
second-seaters.  Included  in  the  plan  was  a  comparison  of  the  combat  performance  of  crews  containing  a 
plot  or  a  navigator  in  the  second  seat. 

Approach 

Data  collection  instruments,  including  a  proficiency  rating  scale,  were  developed  by  personnel 
measurement  psychologists  following  in-depth  interviews  with  F-4  instructor  pilots.  The  purpose  of  the 
proficiency  rating  scale  was  to  measure  differences  betv  een  pilot  and  navigator  second-seaters  in  the 
proficiency  with  which  they  perform  certain  critical  inflight  functions  during  combat  missions. 

The  proficiency  rating  scale  was  administered  at  two  Southeast  Asia  air  bases  where  equal  numbers  of 
newly  trained  pilot  and  navigator  second-seaters  were  assigned.  In  making  ratings,  aircraft  commanders 
compared  their  pilot  or  navigator  crewmember  with  F4  second-seaters  of  equal  combat  experience. 
Information  was  also  gathered  from  intelligence  and  crew  debriefings  concerning  the  degree  to  which 
mission  objectives  were  achieved.  During  the  data  collection  period,  combat  missions  were  limited  to  South 
Vietnam  and  the  southern  portion  of  North  Vietnam. 

Results 

There  was  no  general  superiority  in  performance  of  one  group  of  second-seaters  (pilots  or  navigators) 
over  the  other  across  the  tasks  rated.  Eariy  in  their  combat  experience,  navigator?  were  rated  as  more 
proficient  in  the  Use  of  Radar,  while  pilots  were  rated  as  more  proficient  in  Understanding  Radio 
Communications  and  Visual  Target  Acquisition.  However,  by  the  time  the  second-seaters  accumulated  the 
experience  of  from  26  to  30  combat  missions,  these  differences  were  sharply  reduced,  except  for  the 
performance  of  the  task  Understanding  Radio  Communications,  in  which  case  pilots  retained  their 
superiority.  There  were  no  instances  recorded  in  over  fourteen  hundred  missions  where  either  a  pilot  or 
navigator  second-seater  endangered  the  safety  of  the  aircraft.  Pilot  second-seatera  had  a  higher  percentage  of 
missions  evaluated  as  successful  than  did  navigators.  The  difference  was  small  but  statistically  significant. 
Further  research  is  required  to  determine  whether  the  differences  in  mission  success  would  also  be 
diminished  by  increased  combat  experience. 

Conclusions 

1.  Performance  characteristics  related  to  crewmember  proficiency  can  be  identified  and  reliably 
measured  in  a  combat  environment. 

2.  There  was  no  general  superiority  of  performance  of  either  pilot  or  navigator  second-seat 
crewmembers,  although  pilots  were  consistently  better  at  Understanding  Radio  Communications. 

3.  The  decrease  in  the  magnitude  of  proficiency  differences  which  occurred  between  pilot  and 
navigator  second-seaters  eariy  in  their  combat  experience  indicates  that  the  differences  are  a  function  of 
differences  in  the  training  of  pilots  and  naviptois.  It  may  be  possible,  therefore,  to  reduce  or  eliminate 
these  differences  through  modifications  in  ciew  training  programs, 

This  summary  was  prepared  by  C.R.  Curran,  Armed  Forces  Radiobioiogy  Research  Institute,  Defense 
Atomic  Support  Agency,  Bethesda,  Maryland. 
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PROFICIENCY  DIFFERENCES  OF  PILOT  AND  NAVIGATOR  F4  SECOND-SEAT 
CREWMEMBERS:  A  SOUTHEAST  ASIA  EVALUATION 


1.  PROBLEM 

Background 

In  mid  1967,  the  Department  of  Defense 
requested  that  the  Air  Force  study  the  feasibility 
of  replacing  {slot  rear-seat  crewmembers  in  F4  air¬ 
craft  with  navigators.  The  present  crew  configura¬ 
tion  of  the  F4  places  an  aircraft  commander  in 
the  front  cockpit  and  another  rated  pilot  in  the 
rear  cockpit.  The  Department  of  Defense  esti¬ 
mated  that  the  use  of  navigators  rather  than  pilots 
would  result  in  savings  exceeding  S400  million 
over  s  5-year  period. 

The  Air  Force  was  concerned  that  such  a 
change  in  training  and  assignment  policies  would 
have  a  detrimental  impact  on  combat  effectiveness 
and  aircraft  safety.  The  rationale  for  the  use  of 
pilots  in  the  rear  cockpit  of  the  F4  is  based  on 
their  ability  to  serve  as  backup  pilots  to  provide 
occasional  relief  for  the  aircraft  commander,  to 
recover  the  aircraft  from  dangerous  attitudes  if  the 
aircraft  commander  experiences  severe  spatial 
disorientation,  and  to  fly  the  aircraft  if  the  aircraft 
commander  becomes  incapacitated.  Both  pilots 
and  navigators  receive  navigational  training  while 
only  pilots  receive  forma!  training  in  handling 
aircraft  controls  and  flight  dynamics.  Therefore,  a 
pilot  has  been  considered  the  best  choice  to 
operate  the  rear  cockpit  systems  and  provide 
support  to  the  aircraft  commander. 

Most  aircraft  controls  are  duplicated  in  both 
cockpits,  but  controls  of  certain  navigational 
systems,  such  as  radar  and  the  inertial  navigation 
system,  are  located  only  in  the  rear  cockpit.  The 
basic  requirement  for  a  second-seater  is  to  operate 
the  navigational  subsystems  effectively  and  to  be 
sufficiently  familiar  with  aircraft  dynamics  and 
mission  requirements  to  provide  timely  support  to 
the  aircraft  commander. 

Although  most  rear-cockpit  pilots  eventually 
upgrade  to  the  position  of  aircraft  commander,  the 
period  of  “apprenticeship”  for  a  second-seater  has 
averaged  two  to  three  years.  Some  pilots  have 
expressed  the  opinion  that  serving  for  a  period  of 
time  in  an  essentially  nonpilot  capacity  may  be 
detrimental  to  pilot  skills,  motivation,  and  overall 
job  performance. 

Operational  Problem 

In  response  to  the  DOD  request,  Headquarters, 
United  States  Air  Force  directed  the  Armament 


Development  and  Test  Center,  Eglin  Air  Force 
Base;- to  conduct  a  test  in  both  the  United  States 
and  Southeast  Asia  to  determine  if  there  are  signif¬ 
icant  differences  in  the  combat  performance  of 
navigator  and  pilot  second-waters  that  would  pre¬ 
clude  the  assignment  of  navigators  to  the  F4  or 
other  two-place  fighter  aircraft.  The  test  was 
named  Project  Combat  Team. 

Technical  Problem 

Overall  performance  in  the  highly  complex, 
high-speed  environment  of  F4  combat  is  multi¬ 
dimensional  in  nature.  Thus,  analysis  of  data  from 
several  behavioral  parameters  is  required.  These 
parameters  indude  differences  in  the  frequency 
with  which  tasks  are  performed  and  the  amount  of 
time  spent  on  the  yarious  tasks,  proficiency  in  the 
accomplishment  of  certain  basic  tasks  and  duties, 
and  the  individual  second-seater’s  ability  to  pro¬ 
vide  timely  and  appropriate  support  to  the  aircraft 
commander. 

One  of  the  objectives  of  Project  Combat  Team 
was  to  measure  the  combat  proficiency  of  newly 
graduated  pilots  and  navigators  who  were  assigned 
to  the  second-seat  positions  in  F4  aircraft.  Atten¬ 
tion  was  focussed  on  several  specific  questions: 
whether  there  were  significant  differences  between 
the  two  groups  in  the  proficiency  with  which  they 
performed  certain  inflight  tasks  and  duties  during 
combat  missions;  if  there  were  differences, 
whether  they  were  maintained  as  combat  experi¬ 
ence  increased;  and  whether  there  were  measurable 
differences  between  the  two  groups  in  accomplish¬ 
ing  mission  success  or  maintaining  aircraft  safety. 

A  contaminating  variable  in  assessing  profi¬ 
ciency  of  second-seaters  was  recognized  in  the  fact 
that  the  performance  of  the  aircraft  commander  is 
the  primary  factor  in  accomplishing  mission  objec¬ 
tives,  responding  to  environmental  threats,  and 
maintaining  aircraft  safety.  The  basic  requirement 
for  the  second-seater  is  to  operate  the  rear-cockpit 
subsystems  effectively  and  to  be  sufficiently  famil¬ 
iar  with  aircraft  dynamics  and  mission  require¬ 
ments  to  provide  timely  support  to  the  aircraft 
commander. 

Since  the  design  of  the  study  called  for  the  use 
of  assessments  and  ratings,  another  methodological 
concern  was  related  to  the  psychometric  character¬ 
istics  of  ratings  and  rating  scales.  Proficiency 
ratings  of  various  types  have  provided  much  of  the 
information  required  for  the  operation  of  many 
personnel  systems.  The  utility  of  proficiency 
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ratings,  evidenced  by  their  widespread  use  for  such 
personnel  actions  as  promotion,  career  progres¬ 
sion,  and  assignments,  is  in  part  offset  by  well 
documented  weaknesses  and  defects  (Tiffin  & 
McCormick,  1965).  The  recognized  deficiencies  of 
rating  scales  include  a  tendency  toward  inflation 
of  ratings  with  repeated  use,  contamination  by 
halo  effects,  and  lack  of  uniformity  in  standards 
between  different  raters.  Such  pervasive  defects 
have  made  it  difficult  to  make  inter-group 
comparisons. 

Ratings  of  performance,  however,  have  certain 
compensating  advantages  (Guilford,  1554).  Studies 
have  shown  that  most  performance  ratings  have  a 
satisfactory  degree  of  validity.  Furthermore, 
ratings  may  be  used  to  assess  job  proficiency  in 
situations  where  adequate  objective  criteria  of  per¬ 
formance  are  lacking.  Ratings  are  also  relatively 
easy  to  administer.  Sources  of  error,  such  as  the 
halo  effect,  can  be  reduced  through  careful  plan¬ 
ning  of  the  rating  scale  format  and  administration 
procedures.  Finally,  it  was  anticipated  that  aircraft 
commanders  would  be  satisfactory  raters  because 
of  their  qualifications  as  experienced  F-4  crew¬ 
members  and  their  ability  to  monitor  and  evaluate 
the  second-seaters  proficiency. 

Scope 

A  number  of  assessment  instruments  were  de¬ 
veloped  as  a  part  of  Project  Combat  Team  to 
provide  comprehensive  data  on  aircrew  behavior 
and  performance  during  combat  missions.  These 
included  an  inflight  task  inventory,  a  proficiency 
rating  scale,  a  mission  background  summary,  and  a 
standardized  post-mission  interview.  This  report 
presents  the  data  obtained  from  the  proficiency 
rating  scale.  Mission  evaluation  data  drawn  from 
the  operational  reports  filed  after  each  combat 
mission  are  also  presented.  In  addition,  the  role  of 
proficiency  ratings  in  contributing  to  an  overall 
assessment  of  operationally  significant  differences 
between  pilot  and  navigator  rear-seat  crew¬ 
members  is  discussed.  A  discussion  of  the  F-4  task 
inventory  is  presented  in  an  earlier  report  (Ratliff, 
Shore,  Chiorini,  &  Curran,  1969). 


It.  METHOD 

Subjects 

After  completion  of  their  respective  under¬ 
graduate  pilot  and  navigator  training  courses,  the 
test  subjects,  12  pilots  and  12  navigators,  were 
assigned  to  F-4  combat  crew  training  at  Davis- 
Mcnthan  Air  Force  Base.  Both  the  pilots  and  navi¬ 


gators  were  selected  from  recent  undergraduate 
pilot  and  navigator  course  graduates  available  for 
assignment. 

Normally,  60  training  missions  are  given  to  each 
crew  at  Davis-Monthan.  A  crew  consists  ofa  newly 
upgraded  aircraft  commander  and  a  new  second- 
sealer  assigned  to  fly  with  him  in  Southeast  Asia. 
Crewmembers  are  normally  trained  together  as  a 
team  to  provide  crew  integrity.  Since  only  second- 
seaters  were  subjects  in  the  present  project,  the 
combat  crew  training  course  was  reduced  to  30 
missions  during  which  second-seatera  were  trained 
by  an  experienced  instructor  pilot  in  the  front  seat 
on  all  flights.  The  added  experience  of  the  instruc¬ 
tor  pilots  was  assumed  to  compensate  for  the 
elimination  of  orientation  flights  and  missions 
which  were  primarily  used  as  training  flights  for 
the  student  aircraft  commander. 

Data  Collection  Forms 

The  data  collection  instruments  were  developed 
at  Davis-Monthan  Air  Force  Base  based  on 
information  obtained  during  a  series  of  interviews 
with  15  F-4  instructor  pilots,  most  of  whom  had 
Southeast  Asia  combat  experience.  These  inter¬ 
views  were  conducted  to  determine  those  inflight 
crew  functions  in  which  second-seat  proficiency 
affects  crew  performance  or  mission  effectiveness, 
and  to  identify  significant  inflight  F-4  crew  tasks. 
The  data  coflection  forms  provided  for  a  profi¬ 
ciency  evaluation  of  the  second-seater,  a  mission 
evaluation,  and  an  operational  report  of  the 
mission. 

AFPT  22-2,  F-4  Second-Seater  Proficiency 
Evaluation.  This  instrument,  shewn  in  Appendix  I, 
was  designed  to  measure  proficiency  in  ten 
second-seat  functions  and  three  general  perform¬ 
ance  characteristics.  The  rating  categories  included 
operation  of  aircraft  systems,  crew  coordination, 
and  personal  characteristics.  Aircraft  commanders 
were  instructed  to  compare  the  performance  of 
test  second-seaters  with  that  of  other  second- 
seaters  with  similar  experience.  These  forms  were 
administered  on  a  trial  basis  for  a  series  of  ten 
training  missions  to  the  second-seaters  and  their 
instructor  pilots  at  Davis-Monthan.  Following  this 
experimental  use  of  the  forms,  the  rating  cate¬ 
gories  were  revised  on  the  basis  of  an  assessment  of 
crew  responses  and  post-flight  crew  interviews.  In 
addition,  the  rating  scale  was  expanded  from  7  to 
1 1  points,  a  change  which  improved  the  reliability 
of  the  instrument. 

AFPT  22-1,  Section  3,  Mission  Evaluation.  The 
mission  evaluation  form  is  shown  in  Appendix  I. 
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Its  purpose  was  to  obtain  an  evaluation  from  both 
the  aircraft  commander  and  the  second-seater  of 
the  degree  to  which  mission  objectives  were 
achieved  and  to  record  the  frequency  and  circum¬ 
stances  In  which  the  sccond-seater  made  out¬ 
standing  contributions  to  or  detracted  from 
mission  effectiveness.  Post-mission  interviews  were 
conducted  to  clarify  or  expand  any  information 
‘recorded  on  the  form. 

OP  REP  4,  Operatbnd  Report.  This  document 
contained  mission  data  that  could  be  used  to  eval¬ 
uate  the  extent  of  damage  to  enemy  defenses, 
supply  lines,  etc.  The  report  was  completed  after 
each  mission  during  the  standard  intelligence 
debriefing  given  to  all  returning  crews. 

Data  Collection  Procedures 

Two  Southeast  Asia  tactical  fighter  wings,  the 
8th  TFW,  Ubon  RTAB,  and  the  366th  TFW,  Da 
Nang  RVNAB,  were  each  assigned  a  group  of  12 
test  second-seaters,  6  pilots  and  6  navigators.  Two 
personnel  measurement  psychologists  were 
assigned  to  each  base  to  collect  data  using  stand¬ 
ardized  collection  procedures.  Prior  to  data  collec¬ 
tion,  the  aircraft  commanders  were  extensively 
briefed  on  the  criteria  which  were  to  be  used  in 
evaluating  the  performance  of  the  second-seater. 
The  Mission  Evaluation  form  (AFPT  22-1 ,  Section 
3)  was  administered  to  both  crewmembers  im¬ 
mediately  following  each  combat  mission.  Both 
the  second-seater  and  the  aircraft  commander 
completed  these  forms  within  an  hour  of  landing. 
The  aircraft  commanders  were  then  interviewed  to 
amplify  information  concerning  the  mission  and 
the  second-seater's  performance.  Each  mission  was 
treated  as  a  separate  data  entry.  Information  con¬ 
tained  in  the  Operational  Report  (OP  REP  4)  sup¬ 
plemented  the  data  in  the  completed  Mission 
Evaluation.  After  each  block  of  five  missions 
flown  by  a  test  second-seater,  the  F-4  Second- 
Seater  Proficiency  Evaluation  (AFPT  22-2)  was 
administered  to  each  aircraft  commander  who  flew 
with  him  during  that  block  of  missions. 

In  completing  a  proficiency  rating,  the  aircraft 
commanders  were  instructed  to  compare  the 
second-seater  being  rated  with  second-seaters  of 
equal  experience  in  terms  of  the  number  of 
combat  missions  flown.  No  entries  were  made  for 
categories  in  which  performance  was  not  observed. 

Both  the  aircraft  commanders  and  the  second- 
seaters  were  assured  that  the  information  obtained 
on  the  various  data  collection  forms  would  be 
handled  in  strictest  confidence  and  would  not 
become  a  part  of  any  personnel  records.  Complete 


data  were  collected  for  the  first  30  to  35  combat 
missions  on  22  of  the  24  test  second-seaters.  One 
pilot  was  listed  as  missing  in  action  after  his  17th 
mission  and  one  navigator  reached  Southeast  Asia 
too  late  to  complete  30  missions  by  the  data 
cutoff  date. 

Differences  between  pilots  and  navigators  on 
each  rating  category  were  tested  by  a  multiple 
linear  regression  analysis  (Bottenberg  &  Ward, 
1963).  Three  specific  questions  were  tested  con¬ 
cerning  each  category:  (a)  Were  there  proficiency 
differences  between  pilot  and  navigator  second- 
seaters  during  the  first  30  combat  missions?  (fe) 
Were  the  magnitudes  of  any  differences  stable  as 
combat  experience  increased?  (c)  Were  there  any 
proficiency  differences  between  pilots  and  navi¬ 
gators  by  the  time  they  had  flown  from  26  to  30 
missions? 

Mission  Evaluation 

A  board  of  12  aircraft  commanders  who  were 
currently  assigned  to  combat  units  in  Southeast 
Asia  was  convened  at  Eglin  Air  Force  Base  to  rate 
the  effectiveness  of  combat  missions  in  which  the 
test  second-seaters  participated  in  Southeast  Asia. 
The  board  was  composed  of  ten  majors  and  two 
lieutenant  colonels  with  recent  combat  experience 
in  the  F-4  ranging  from  50  to  150  missions.  Infor¬ 
mation  available  to  the  board  for  each  mission 
rated  included  second-seater  and  aircraft  com¬ 
mander  responses  to  the  mission  evaluation 
portion  of  AFPT  22-1,  a  copy  of  the  Operational 
Report,  and  copies  of  post-mission  interviews  with 
the  aircraft  commanders.  The  board  had  no 
information  as  to  crew  composition  for  any  of  the 
missions  rated. 

The  12  board  members  were  divided  into  two 
working  groups  and  instructed  to  rate  the  missions 
according  to  the  extent  to  which  they  were  accom¬ 
plished  in  accordance  with  the  target  description 
appearing  on  the  Operational  Report.  They 
worked  individually  and  rated  the  missions  in 
terms  of  four  possible  alternatives:  all  mission 
objectives  achieved,  some  mission  objectives 
achieved,  no  mission  objectives  achieved,  or  insuf¬ 
ficient  information  to  rate  the  mission.  If  fewer 
than  four  of  the  six  judges  agreed  upon  a  rating, 
the  mission  was  discussed  by  the  six-man  group.  If 
agreement  was  not  reached  in  this  conference,  the 
mission  was  rated  as  non-scorable. 

Data  Restrictions 

Restriction  of  missions  to  South  Vietnam  and 
the  southern  portion  of  North  Vietnam  during  the 
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entire  Southeast  Asia  data  collection  period 
limited  the  spectrum  of  missions  upon  which  data 
could  be  collected.  Missions  over  the  most  heavily 
defended  areas  of  North  Vietnam  and  observations 
of  the  test  subjects  reacting  to  MIG  or  SAM 
threats  were  not  available  for  this  report.  Any 
interpretation  of  the  data  should  take  these  limita¬ 
tions  into  account. 


111.  RESULTS 

The  results  of  the  second-seater  proficiency 
ratings  made  by  the  aircraft  commanders  are 
shown  in  Figure  1.  The  results  of  the  statistical 
analyses  are  presented  in  Appendix  II.  Navigators 
were  rated  substantially  higher  than  pilots  on 
Category  2,  Use  of  Radar  (p  <  .01).  This  differ¬ 
ence  was  considerably  reduced  by  the  26-to-30 
mission  segment,  and  group  differences  at  this 
point  were  not  statistically  significant.  Pilots  were 
rated  as  more  proficient  than  navigators  on  Cate¬ 
gory  5,  Visual  Target  Acquisition  and  Identifica¬ 
tion  (p  <  .03).  Differences  at  the  26-to-30  mission 
segment  were  not  statistically  significant.  Pilots 
were  also  rated  higher  (p  <  .07)  than  navigators  on 
Category  4,  Understanding  Radio  Communica¬ 
tions,  and  this  group  difference  was  significant  at 
the  26-to-30  mission  segment  (p  <  .04).  Category 
7,  Calling  Out  Correct  Action  in  Response  to  a 
Threat,  and  Category  8,  Emergency  Situations, 
were  not  analyzed  since  second-seaters  were  infre¬ 
quently  rated  on  these  categories. 

An  estimate  of  the  reliability  of  the  ratings  was 
calculated  by  correlating  the  mean  of  the  first, 
third,  and  fifth  ratings  on  each  category  with  the 
mean  of  the  second,  fourth,  and  sixth  ratings. 
These  figures  are  presented  in  Table  1.  Categories 
7  and  8  are  not  presented  due  to  insufficient  data. 

In  order  to  determine  the  sensitivity  of  the 
measurement  instrument,  separate  computations 
of  means  were  made  for  pilots  and  navigators  by 
base  of  assignment.  Whereas  pilots  had  been  as¬ 
signed  to  the  two  Southeast  Asia  bases  without 
regard  to  standings  in  their  combat  crew  training 
class,  navigators  with  the  highest  class  standings 
were  assigned  to  Ubon,  and  those  with  the  lowest 
were  assigned  to  Da  Nang.  The  method  of  assign¬ 
ment  provided  an  opportunity  to  observe  the  test 
instrument’s  sensitivity  in  detecting  the  difference 
in  the  two  navigator  groups.  Figures  2  and  3 


present  the  mean  proficiency  ratings  by  base.  Cate¬ 
gories  7  and  8  are  not  presented  due  to  insufficient 
data.  Ubon  navigators  were  rated  superior  to  Da 
Nang  navigators  on  all  categories,  and  the  differ¬ 
ences  remained  consistent  for  the  entire  data  col¬ 
lection  period.  On  the  other  hand,  there  were  no 
consistent  differences  between  the  two  groups  of 
pilots. 

Additional  information  concerning  second- 
seater  proficiency  was  obtained  from  the  Mission 
Evaluation,  AFPT  22-1,  Section  3.  In  question  4, 
both  the  second-seater  and  his  aircraft  commander 
were  asked  to  identify  second-seater  actions  which 
required  corrective  action  by  the  aircraft  com¬ 
mander.  These  actions  included  second-seater 
errors  in  equipment  operation,  failure  to  make 
radio  frequency  changes,  etc.  In  question  5,  infor¬ 
mation  was  requested  from  both  the  second-seater 
and  his  aircraft  commander  concerning  instances 
during  the  mission  when  the  second-seater  pro¬ 
vided  outstanding  support  to  the  aircraft  com¬ 
mander.  Comments  here  ranged  from  unqualified 
statements  (e.g.,  “excellent  support”)  to  specific 
instances,  such  as  outstanding  use  of  particular 
pieces  of  equipment  under  stressful  conditions. 
Responses  to  question  4  were  tallied  when  either 
the  aircraft  commander  or  the  second-seater 
responded,  whereas  only  the  aircraft  commanders’ 
responses  to  question  5  were  scored.  The  data 
from  questions  4  and  5  are  presented  in  Table  2. 

Both  pilots  and  navigators  showed  decreases 
over  time  in  the  percentage  of  missions  on  which 
their  actions  required  correction  or  compensation 
by  their  aircraft  commanders.  Pilots  showed  a 
more  rapid  decrease  and  had  an  overall  lower  per¬ 
centage  of  such  missions.  Responses  to  question  5 
showed  little  differentiation  between  the  two 
groups,  and  there  were  no  consistent  changes  in 
the  percentages  of  favorable  comments  over  time. 

The  comparisons  between  pilots  and  navigators 
on  mission  effectiveness  over  all  30  missions  made 
by  the  board  of  1 2  aircraft  commanders  are  sum¬ 
marized  in  Table  3.  Approximately  the  same 
number  of  missions  was  scored  for  both  pilots  and 
navigators.  The  data  were  analyzed  in  two  cate¬ 
gories:  all  mission  objectives  achieved,  and  less 
than  all  objectives  achieved.  Pilots  showed  signifi¬ 
cantly  more  successful  and  fewer  unsuccessful 
missions  than  did  navigators  ( p  <  .05). 


4 


1-  6-  11-  16-  21- 

5  10  16  20  25 


26- 

30 


MISSION  BLOCK 


Fig.  2.  Navigator  sccond-seater  group  ratings  on  proficiency 
averaged  over  5-mission  blocks. 
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Fig.  I  Pilot  second-aeater  group  ratings  on  proficiency  categories  by  base  of  assignment,  averaged 
over  5-mission  blocks. 
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Table  1.  Reliability  Coefficients  for  Rating 
Categories  of  Second-Sester 
Piofidesey  Evaluation 
(Rating  Categories  from  A  FIT  22-2,  F-4 
Second-Scoter  Proficiency  Evaluation) 


Ratios  Category 

RaHabMty 

Coefficient 

1 

Mission  Preparation 

.83 

2 

Use  of  Radar 

.78 

3 

Navigation 

85 

4 

Understanding  Radio  Com- 

munications 

.70 

5 

Visual  Target  Acquisition 

and  Identification 

.48 

6 

Visual  Defensive  Surveillance 

.67 

7a 

Calling  Out  Correct  Action 

in  Response  to  a  Threat 

- 

8a 

Emergency  Situations 

• 

9 

Reaction  to  Combat  Stress 

.47 

10 

Staying  Ahead  of  Situation  and 
Keeping  Aircraft  Commander 

Informed  During  Mission 

.67 

11 

Job  Knowledge 

.69 

12 

Attitude  Toward  Job 

Performance 

.47 

13 

Overall  Performance 

.34 

aReliab8ities  for  Categories  7  and  8  were  not  computed 
because  of  insufficient  data. 


Table  2.  Percentage  of  Missions  on 
Which  the  Second-Seat  er’s  Actions 
Detracted  from  or  Contributed 
Outstandingly  to  Mission 
Effectiveness 

(Computed  from  Responses  to  Questions  4*  and  5b 
on  the  Mission  Evaluation,  AFPT  22-1,  Section  3.) 


Mlstlsn  Whan 

Action  Occurred 

Percentage  of  Mission* 
Whan  Action  Occurred 

Navigator* 

Pilot* 

Actions  Required  Compensation 

Missions  1  *10 

20.8 

17.5 

Missions  11-20 

17.5 

6.8 

Missions  21-30 

13.3 

4.5 

All  Missions 

Combined 

17.2 

9.8 

Actions  Provided  Outstanding  Support 

Missions  1  *10 

30.0 

25.0 

Missions  11-20 

25.0 

20.0 

Missions  21*30 

30.0 

39.1 

All  Missions 

Combined 

28.3 

28.0 

aQuestion  4.  Did  second-seat  crewmember’s  perform¬ 
ance  detract  from  mission  effectiveness  or  require 
compensation  by  the  aircraft  commander? 

bQueslion  5.  Were  there  instances  when  the  second- 
seht  crewmember  provided  outstanding  support  to  the  air¬ 
craft  commander  or  to  mission  effectiveness? 


Table  3.  Mission  Effectiveness 
Judgments  by  Board  of  Aircraft 
Commanders 

(Judgments  Based  on  Data  from  the  Mission 
Evaluation,  AFPT  22-1,  Section  3) 


Million 
Effective  ness 
Evaluation 

Million*  with 
Navigator 
Second-Seater* 

N  (Percent) 

Million*  with 
Pilot 

Second-Seater* 

N  '(Percent) 

All  Objectives 
Achieved 

305 

(87) 

319 

(92) 

All  Objectives 
Not  Achieved 

45 

(13) 

27 

(  8) 

X2  = 
P  < 

4.79 

.05 
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IV.  DISCUSSION 

The  proficiency  ratings  showed  no  general 
superiority  in  performance  of  one  group  of 
second-seaters  over  the  other.  Specifically,  navi¬ 
gators  received  higher  proficiency  ratings  than 
pilots  in  the  Use  of  Radar  while  pilots  received 
higher  ratings  in  Understanding  Radio  Communi¬ 
cations  and  Visual  Target  Acquisition.  However, 
the  important  finding  for  operational  considera¬ 
tion  is  that,  of  the  few  significant  differences 
found  between  pilot  and  navigator  second-seaters, 
only  in  Understanding  Radio  Communications  was 
a  difference  maintained  to  the  26-to-30  mission 
segment. 

If  the  observed  proficiency  differences  between 
pilot  and  navigator  second-seaters  had  been  main¬ 
tained  over  the  30  missions  sampled,  such  differ¬ 
ences  might  have  been  related  either  to  training 
differences  or  to  selection  differences.  However, 
since  the  proficiency  differences  decreased  sharply 
over  the  first  30  combat  missions,  it  can  be  con¬ 
cluded  that  proficiency  differences  between  pilots 
and  navigators  can  be  reduced  or  eliminated  as  a 
result  of  modifications  in  crew  training  programs. 

The  method  used  to  inspect  the  sensitivity  of 
the  ratings  compares  the  ratings  obtained  in  com¬ 
bat  with  rankings  made  earlier  for  performance 
during  training.  Specifically,  the  earlier  rankings 
were  the  class  standings  of  the  second-seaters  at 
the  completion  of  combat  crew  training.  Navi¬ 
gators  assigned  to  Ubon  were  all  ranked  higher 
than  navigators  assigned  to  Da  Nang,  and  their 
combat  proficiency  ratings  consistently  discrim¬ 
inated  between  these  two  groups.  Pilots  at  the  two 
bases  were  similar  in  terms  of  dass  standings  and 
combat  proficiency  ratings.  Thus,  when  real  differ¬ 
ences  were  known  to  exist  in  the  proficiency  of 
second-seaters,  differences  appeared  in  the  profi¬ 
ciency  ratings  which  were  consistent  with  pre¬ 
dictions  that  could  be  made  from  the  nature  of  the 
previously  known  group  disparities.  To  the  extent 
that  the  class  standings  from  combat  crew  training 
are  a  valid  measure  of  performance,  then  the  group 
performance  curves  by  base  provide  an  index  of 
the  validity  of  the  rating  categories  used  in  the 
proficiency  evaluation  instrument  (AFPT  22-2). 

The  reliability  of  the  proficiency  data  was  esti¬ 
mated  by  calculating  reliability  coefficients  com¬ 
paring  the  mean  of  the  odd-  and  even-numbered 
mission  blocks.  Because  the  proficiency  of  both 


the  pilots  and  the  navigators  was  increasing  over 
the  first  30  missions,  the  reliability  coefficients 
calculated  for  each  rating  category  were  con¬ 
sidered  to  be  conservative.  However,  the  transi¬ 
tional  nature  of  the  behavior  being  rated  in  the 
present  study  made  more  traditional  methods  for 
computing  reliability  coefficients  less  appropriate. 
Other  factors  which  may  have  influenced  the 
reliability  coefficients  include  the  large  number  of 
judges  that  had  to  be  used  to  rate  the  test  second- 
seaters,  the  wide  range  of  mission  requirements 
between  sorties,  and  the  large  variations  in  combat 
stress  from  mission  to  mission. 

The  validity  and  reliability  demonstrated  in  the 
data  answered  a  basic  question  raised  by  Project 
Combat  Team.  Namely,  can  objective,  reliable  pro¬ 
ficiency  ratings  be  collected  when  complex  inter¬ 
personal  relationships  are  established  between 
rater  and  ratee  in  a  highly  stressful  environment? 
Since  the  observed  psychometric  characteristics  of 
the  ratings  obtained  in  this  study  were  comparable 
to  those  of  rating  data  collected  under  laboratory 
conditions  (Guilford,  3954,  Tiffin  &  McCormick, 
1965),  it  appears  that  it  is  possible  to  collect  use¬ 
ful  ratings  of  proficiency  under  combat  con¬ 
ditions. 

Previous  work  with  rating  scales  has  concen¬ 
trated  on  the  analysis  of  task  or  job  proficiency 
for  individuals  or  for  homogeneous  groups  of 
individuals.  The  present  data  represent  a  signifi¬ 
cant  departure  from  previous  work  in  that  the  job 
was  held  generally  constant  while  the  proficiency 
of  two  groups  with  known  differences  in  their 
training  backgrounds  was  inspected.  Thus,  it  was 
possible  to  inspect  the  differences  in  proficiency  at 
a  certain  job  (F4  second  seat)  between  two  groups 
in  a  strictly  operational  setting.  The  unique  experi¬ 
mental  plan  for  Project  Combat  Team,  requiring 
multidimensional  behavioral  data,  also  provided  an 
opportunity  to  compare  the  results  of  the  profi¬ 
ciency  ratings  of  the  two  groups  with  other  per¬ 
formance  data  obtained  on  these  groups  using  a 
task  inventory.  Inconsistencies  in  the  results  of 
these  two  performance  evaluations  would  have 
weakened  confidence  in  the  data  from  either  or 
both  instruments.  However,  the  findings  of  the 
proficiency  ratings  of  pilot  and  navigator  second- 
seaters  were  completely  consistent  with  the 
findings  of  the  task  inventory  data.  A  more 
detailed  discussion  may  be  found  in  an  earlier 
report  on  the  study  (Ratliff  et  ah',  1969). 
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The  data  collected  from  pest-mission  interviews 
(AFPT  22-1,  Pan  3)  and  intelligence  debriefing  re¬ 
ports  (OP  REP  4)  showed  that,  over  the  first  30 
combat  missions,  pilot  second-sealers  flew  on 
more  missions  rated  completely  successful  and 
fewer  rated  as  not  meeting  all  the  mission  objec¬ 
tives  than  did  the  navigators;  the  differences  were 
small  but  statistically  significant. 

The  specific  contribution  to  mission  effec¬ 
tiveness  of  those  tasks  for  which  proficiency 
ratings  were  made  could  not  be  objectively  eval¬ 
uated.  Thus,  the  judgment  of  individuals  respon¬ 
sible  for  operational  decision-making  at  the  upper 
levels  of  management  must  be  relied  upon  to 
identify  the  causal  linkage  between  proficiency  at 
certain  inflight  tasks  and  combat  mission  success. 
A  discussion  of  the  possible  ways  the  proficiency, 
mission  evaluation,  and  task  analysis  data  collected 
for  Project  Combat  Team  nay  be  integrated  by 
management  personnel  can  be  found  in  Ratliff  et 
d.  (1969). 

The  reliability  of  the  proficiency  data  may  have 
been  improved  if  the  proficiency  evaluation  form 
had  been  administered  during  the  post-mission 
interview  immediately  after  each  mission  rather 
than  after  each  block  of  five  missions.  The  latter 
procedure  was  established,  however,  to  minimize 
the  data  collection  tins  required  of  crewmembers 
immediately  following  each  combat  mission. 
Another  way  in  which  the  psychometric  character¬ 
istics  of  proficiency  data  collected  under  combat 
conditions  may  be  improved  is  to  include  the 
proficiency  categories  at  appropriate  locations  in 
the  task  analysis  form.  This  can  be  done  without 
appreciably  lengthening  the  amount  of  time  taken 
by  the  post-mission  interview.  In  a  study  which 
tested  such  a  procedure  (Tiffin  &  McCormick, 
1965),  it  was  found  that  proficiency  information 
collected  in  this  manner  had  higher  coefficients  of 
reliability  than  proficiency  data  coilected  under 
identical  conditions  on  a  separate  form. 


V.  SUMMARY  AND  CONCLUSIONS 

In  response  to  a  request  from  the  Department 
of  Defense,  the  Air  Force  established  Project 
Combat  Team  to  study  the  feasibility  of  replacing 
pilots  with  navigators  in  the  rear  cockpit  of  F-4 
fighter  aircraft.  In  the  absence  of  objective  per¬ 


formance  criteria  in  the  combat  setting,  it  was 
determined  that  data  should  be  collected  using 
several  performance  parameteis.  For  this  study, 
data  were  collected  to  answer  three  questions:  (u) 
Are  there  significant  differences  between  pilot  and 
navigator  second-seaters  in  the  proficiency  with 
which  they  perform  certain  inflight  tasks  and 
duties  during  combat  missions?  (6)  If  found,  are 
differences  in  proficiency  maintained  as  combat 
experience  increases?  (c)  Do  differences  in  training 
background  result  in  measurable  differences  in 
mission  success  or  aircraft  safety?  Since  aircrew 
proficiency  nad  never  been  measured  under 
combat  conditions,  the  data  were  also  analyzed  to 
determine  if  it  is  feasible  to  collect  reliable  profi¬ 
ciency  measures  in  a  combat  environment. 

Analyses  of  the  proficiency  rating  information 
included  pilot  and  navigator  group  means  for  six 
successive  five-mission  blocks  on  13  proficiency 
categories,  group  mean  data  plotted  as  a  function 
of  Southeast  Asian  base  of  assignment,  and  esti¬ 
mates  of  test-retest  reliability.  Mission  evaluation 
data  and  operational  report  information  generated 
during  the  standard  intelligence  debriefing  were 
also  presented  in  order  to  inspect  the  success  of 
combat  missions  flown  with  pilots  or  navigators 
occupying  the  second  seat  of  the  aircraft. 

These  data,  although  collected  in  a  limited 
combat  environment,  lead  to  several  conclusions 
with  regard  to  some  methodological  questions 
involving  the  data  collection  instruments,  as  well 
as  to  the  more  specific  questions  of  pilot  and  navi¬ 
gator  performance  differences. 

1 .  It  is  possible  to  obtain  proficiency  data  in  a 
combat  environment  that  are  sufficiently  reliable 
to  be  used  by  higher  management  in  determining 
the  crew  manning  requirements  of  F4  aircraft. 

2.  It  is  possible  to  use  ratings  of  task  or  job 
proficiency  to  measure  the  differences  between 
two  groups  in  an  operational  setting. 

3.  The  proficiency  rating  scale  used  in  the 
present  study  was  sensitive  enough  to  reflect  group 
disparities  which  were  previously  known  to  exist 
in  the  pilot  and  navigator  subjects. 

4.  The  proficiency  ratings  showed  no  general 
superiority  in  performance  of  one  group  of 
second-seaters  over  the  other. 
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5.  Early  in  their  combat  experience,  navies  ton 
received  higher  proficiency  rating  in  the  Use  of 
Radar  while  plots  received  higher  ratings  in 
Understanding  Radio  Communications  end  Visual 
Target  Acquisition.  However,  the  important  opera¬ 
tional  finding  was  that  these  eady  difference  were 
sharply  reduced  by  the  iin»  they  had  flown  26  to 
30  missions,  except  in  the  case  of  Understanding 
Radio  Communications . 

6.  The  transitional  future  of  the  proficiency 
differences  indicates  that  they  are  a  fimcfion  of 


pilot  and  navigator  training  differences  and  can, 
therefore,  be  reduced  or  eliminated  through  modi¬ 
fications  in  crew  training  programs. 

7.  During  the  first  30  combat  missions,  pilot 
second-aeaters  flew  on  more  missions  rated  com¬ 
pletely  successful  than  did  navigators,  the  differ¬ 
ences  were  small  but  statistically  significant. 
However,  these  differences  in  mission  success 
could  not  be  objectively  related  to  proficiency  dif¬ 
ferences  between  pilots  and  navigators. 
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APPENDIX  I.  DATA  COLLECTION  INSTRUMENTS5 

DO  NOT  WRITE  IN  THESE  SPACES 


GIB 

A/C 

WHO 

#GIB  1 

MSNS 

CELL 

■ 

■ 

i 

I 

■ 

■ 

□ 

ft 

F4GIB 

PROFICIENCY  EVALUATION 

A/C-  Consider  the  GIB’s  performance  on  ail  missions  he  has  flown  with  you  since  your  last  evaluation  of 
him.  Rate  him  relative  to  other  GIBs  with  a  similar  number  of  combat  missions. 

GIB-  Consider  your  own  performance  on  all  missions  since  you  last  made  this  evaluation.  Rate  yourself 
relative  to  other  GIBs  with  a  similar  number  of  combat  missions. 


■ 

2  3  4 

5  6 

7  8  9 

10 

1- 

11 

below  average 

aveiaee 

above  averaae 

°  °  °  | 

Enter  the  appropriate  number  for  a  GIB  rating  on  each  of  the  following  factors. 
RATE  PERFORMANCE  IN:  (Consider  factors  observed  since  last  evaluation) 

1 .  Mission  preparation . 

2.  Use  of  radar . 

3.  Navigation . 

4.  Understanding  radio  communications  . 

5.  Visual  target  acquisition  and  identification . 

6.  Visual  defensive  surveillance  . 

7.  Calling  out  correct  action  in  response  to  a  threat  . 

8.  Emergency  situations  . . 

9.  Reaction  to  combat  stress . . . 

10.  Staying  ahead  of  the  situation  and  keeping  the  A/C 

informed  during  mission . 


*The  term  GIB,  or  “Guy  in  bade,"  used  in  the  data  collection  instruments  is  the  operational  vernacular  for  the 
second-seat  crewmember. 
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F4  GIB  Proficiency  Evaluation  (Continued) 


RATE  GIB  ON  THESE  GENERAL  FACTORS: 

1.  JOB  KNOWLEDGE  . 

2.  ATTITUDE  TOWARD  JOB  PERFORMANCE  ...... 

t**tt«***0»«9#»**0.) 

RATE  GIB’s  OVERALL  PERFORMANCE . 

COMMENTS:  (Strengths,  weaknesses,  other) 


AFPT  22-2 
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DO  NOT  WRITE  IN  THESE  SPACES 


GIB  A/C  WHO 

m  m  n 


MSN# 

PKG  # 

u 

□ 

— 

□ 

MSN  TYPE 


MISSION  EVALUATION 

The  answers  to  the  following  questions  will  be  kept  strictly 
confidential.  It  is  important  that  the  questions  be  answered  with 
complete  frankness.  This  material  wfll  NOT  be  reflected  in  any 
report  about  a  particular  GIB  or  A/C  but  will  be  essential  in 
helping  us  achieve  the  object  of  our  study. 


1.  If  the  GIB  performed  flying  duties  during  any  part  of  the  mission  (flight  formation,  recovery  from 
unusual  attitude,  combat  maneuvers,  defensive  maneuvers  in  response  to  MIGs,  SAMs,  and  AAA,  letdown, 
etc.),  please  describe  the  situation  and  tell  why  GIB  handled  the  flight  controls. 


2.  Rate  effectiveness  of  mission  as  planned: 

(  )  a.  All  objectives  achieved. 

(  )  b.  Son.?  objectives  achieved. 

(  )  c.  No  objectives  achieved. 

Explain  b  or  c: 


3.  Rate  effectiveness  of  mission  as  it  was  flown:  (Consider  changes  in  plans  as  mission  progressed). 

(  )  a.  All  objectives  achieved. 

(  )  b.  Some  objectives  achieved. 

(  )  c.  No  objectives  achieved. 

Explain  a,  b,  or  c: _ 


4.  Did  GIB’s  performance  detract  from  mission  effectiveness  or  require  compensation  by  the  A/C? 
(  )  No 

(  )  Yes  -  Briefly  explain: _ _ 


5.  Were  there  instances  when  the  GIB  provided  outstanding  support  to  the  aircraft  commander  or  to 
mission  effectiveness? 

(  . )  No 

(  )  Yes  -  Briefly  explain: _ _ 


6.  COMMENTS: 


AFPT22-1 


15 


APPENDIX  II.  MULTIPLE  LINEAR  REGRESSION  ANALYSIS  OF  PROFICIENCY  DATA 


PREDICTOR  VARIABLES 

For  the  regression  analyses,  34  predictor  variables  were  generated.  Equations  were  solved  for  each  of 
the  performance  variables  using  various  combinations  of  these  predictors. 


Pradletcr 


Variable 

DotcrlpUon 

Scora  Rango 

X, 

Pilot 

X2 

Navigator 

0,1 

X3 

1  -5  Mission  Block 

0,1 

X4 

6-10  Mission  Block 

0,1 

X5 

11-15  Mission  Block 

0,1 

X6 

16-20  Mission  Block 

0,1 

X, 

21-25  Mission  Block 

0,1 

X8 

26-30  Mission  Block 

0,1 

X, 

Pilot  x  1-5  Mission  Block 

0,1 

X,  0 

Pilot  x  6-10  Mission  Block 

0,1 

Xu 

Pilot  x  11-15  Mission  Block 

0,1 

x„ 

Pilot  x  16-20  Mission  Block 

0,1 

Xl  3 

Pilot  x  21-25  Mission  Block 

0,1 

X|  4 

Pilot  x  26-30  Mission  Block  t 

0,1 

X,s 

Navigator  x  1-5  Mission  Block 

0,1 

X,  6 

Navigator  x  6-10  Mission  Block 

0,1 

X,  7 

Navigator  x  1 1-15  Mission  Block 

0,1 

x,  8 

Navigator  x  16-20  Mission  Biock 

0,1 

x,9 

Navigator  x  21-25  Mission  Block 

0,1 

X30 

Navigator  x  26-30  Mission  Block 

0,1 

Xj 1 

Pilot  x  1-5  Mission  Block  +  2(Pilot  x  6-10  Mission  Block) 

+  3(Pilot  x  1 1-15  Mission  Block)  +  4(Pilot  x  16-20  Mission 

Block)  +  5(Pilot  x  21-25  Mission  Block)  +  6{Pilot  x 

26-30  Mission  Block) 

X;  1 

Navigator  x  1-5  Mission  Block  +  2(Navigator  x  6-10  Mission  Block) 
+  3(Navigator  x  11-15  Mission  Block) 

+  4{Navigator  x  16-20  Mission  Block)  +  5{Navigator 
x  21-25  Mission  Block)  +  6{Navigator  x  26-30  Mission  Block) 

0,1,. 

.,21 

Xj  3 

(X51)2 

0,1,. 

.,4^1 

X24 

(X2  2  )2 

0,1,. 

..441 

Xjs 

Xj,  +X22 

0,1,. 

.,42 

Xj6 

X23  +  X24 

0,1,. 

<N 

OO 

OO 

X2  7 

X,  +  X22 

0,1, . 

..,22 

Xj8 

X|  +  X24 

0,1,. 

.,442 

Xj9 

Xj,  -X, 

0,1,. 

.,20 

X30 

Xj3-X, 

0,1,. 

.,440 

X3 1 

X2j  +  6X, 

0,1,. 

.,27 

X32 

XJ4  +36X, 

0,1,. 

.,447 

X33 

Xj, -6X, 

0,1,. 

.,15 

X34 

X2  3  -36X, 

0,1,. 

.405 
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HYPOTHESES 


Of  the  various  hypotheses  tested,  six  were  of  particular  interest  in  this  study.  For  each  of  the 
performance  variables,  the  following  questions  were  analyzed  to  assess  pilot  and  navigator  performance 
differences. 


1.  Are  the  magnitudes  of  differences  between  pilot  and  navigator  second-seat 

crewmembers  stable  as  combat  experience  increases?  (Analysis  using  corpmon  weight  on  linear 
and  quadratic  cel!  number  vectors) 

2.  During  the  first  30  missions,  is  there  an  overall  difference  between  the  proficiency  of 
pilot  and  naviptor  second-seat  crewmembers?  (Analysis  using  both  linear  and  quadratic  weights 
on  cell  number  vectors) 

3.  Are  the  magnitudes  of  differences  between  pilot  and  naviptor  second-seat 

crewmembers  stable  as  combat  experience  increases?  (Analysis  using  common  weight  on  linear 
cel!  number  vectors) 

4.  During  the  first  30  missions,  is  there  an  overall  difference  between  the  proficiency  of 
pilot  and  naviptor  second-seat  crewmembers?  (Analysis  using  linear  weights  on  cell  number 
vectors) 

5.  Are  there  proficiency  differences  between  pilot  and  navigator  second-seat 

crewmembers  during  their  first  five  combat  missions? 

6.  Are  there  proficiency  differences  between  pilot  and  navigator  second-seat 

crewmembers  during  their  26th  to  30th  missions? 
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RESULTS 


Results  of  the  analyses  for  the  performance  variables  are  presented  tn  Table  4.  For  each  variable  (i.e., 
proficiency  category),  results  of  the  F  tests  for  the  six  hypotheses  are  presented. 


Table  4.  Results  of  Multiple  Regression  Analysis  and  F  Tests  of  Significance 
for  Proficiency  Data 


Hypcth- 

Bill 

R 2 
FuS 

Model 

R7 

Restricted 

Model 

dfl 

dfi 

F 

SlflRifl* 

eanc» 

Levt! 

Hypo  th¬ 
em 

* 

Fun 

Model 

R7 

Restricted 

Model 

<Vt 

<*h 

F 

5l*nl«- 

cjr.ce 

Level 

Category  1:  Mission  Preparation 

Category  2: 

Use  of  Radar 

1 

.1633 

.1493 

2 

131 

1.0911 

33889 

1 

.1564 

.1364 

2 

130 

1.5382 

.21865 

2 

.1493 

.1457 

1 

133 

0.5685 

.45220 

2 

.1364 

.0921 

1 

132 

6.7664 

.01035 

3 

.1502 

.1493 

1 

133 

0.13S8 

.71003 

3 

.1515 

.1362 

1 

132 

2.3761 

.12560 

4 

.1493 

.1457 

1 

134 

0.5726 

.45055 

4 

.1362 

.0920 

1 

133 

6.8167 

.01007 

5 

.1633 

.1572 

1 

131 

0.9500 

.33151 

5 

.1564 

.1373 

1 

130 

2.9442 

.08858 

6 

.1633 

.1495 

1 

131 

2.1633 

.14373 

6 

.1564 

.1554 

1 

130 

0.1496 

.69953 

Category  3: 

Navigation 

Category  4:  Understanding  Radio  Communications 

1 

.1302 

.1141 

2 

131 

1.2132 

.30055 

1 

.2070 

.1931 

2 

131 

1.1450 

32138 

2 

.1141 

.1000 

1 

133 

2.1218 

.14758 

2 

.1931 

.1732 

1 

133 

3.2819 

.07231 

3 

.1122 

.1115 

1 

133 

0.1084 

74246 

3 

.1948 

1928 

1 

333 

0.3352 

.56357 

4 

.1115 

.0972 

1 

134 

2.1582 

.14415 

4 

1928 

.1728 

1 

134 

3.3202 

.07065 

S 

.1302 

1298 

1 

131 

0.0637 

.801 14 

5 

2070 

1967 

1 

131 

1.6958 

.19511 

6 

.1302 

.1285 

1 

131 

0.2562 

61358 

6 

2070 

1816 

1 

131 

4.1904 

.04265 

Category  5:  Visual  Target  Acquisition 

and  identification  Category  6:  Visual  Defensive  Surveillance 


1 

1096 

1084 

2 

129 

0  0859 

91772 

1 

1536 

1447 

2 

118 

0  6205 

53945 

2 

1084 

0765 

1 

131 

4  6936 

03208 

2 

1447 

1401 

1 

120 

0.6457 

42323 

3 

1083 

1071 

1 

131 

0  1790 

67290 

3 

1457 

1441 

1 

120 

0,2285 

.63352 

4 

1071 

0752 

1 

132 

4  7150 

03169 

4 

i  44 1 

1394 

1 

121 

0.6597 

41828 

J 

1096 

1050 

1 

129 

0  6600 

41808 

5 

1536 

1506 

1 

118 

0.4168 

51977 

6 

1096 

0976 

1 

129 

!  7439 

18898 

6 

1356 

1418 

1 

118 

1  6465 

20194 

IK 


Table  4  ( Continued ) 


€§& 

R2 

f'uE 

Mt>*8 

awtrtotw? 

4A 

d/j 

r 

stsam- 

c*nc* 

L*V« 

Hypelh. 

etis 

P.2 

Was 

Model 

R2 

Model 

<*/i 

p 

s»rnlft> 

cence 

Level 

Category  7:  CgSiag  Out  Correct  Action 

la  Response  to  a  Threat 

Category  8:  Emergency  Situations 

1 

.2238 

.0672 

2 

96 

3.5043 

.03397 

1 

.1174 

.1160 

2 

77 

0.0593 

.94242 

2 

.1672 

.1434 

1 

98 

2.7919 

,09794 

2 

.1160 

.0944 

1 

79 

1.9346 

.16816 

3 

.1703 

.1635 

1 

98 

0.7988 

.37365 

3 

.0713 

.0706 

1 

79 

0.0664 

.79730 

4 

.1635 

.1338 

i 

99 

2.9240 

.09041 

4 

.0706 

.0545 

1 

80 

13835 

.24300 

5 

.2238 

.2014 

1 

96 

2.7686 

.09939 

5 

.1174 

.1097 

1 

77 

0.6730 

.41455 

6 

.2238 

.1569 

I 

96 

8.2826 

.00493 

6 

.1174 

.1144 

1 

77 

0.2618 

.61033 

Category  10: 

Staying  Ahead  of  the  Situation  and 

Category  9:  Reaction  to  Combat  Stress 

Keeping  the  A/C  Informed  During  the  Mission 

1 

.1353 

.1192 

2 

130 

1.2078 

3021 9 

1 

.1047 

.0883 

2 

131 

1.2046 

.30311 

2 

.1192 

.1183 

1 

132 

0.1374 

.71147 

2 

.0383 

.0774 

1 

133 

1.5916 

.20932 

3 

.1215 

.1190 

1 

132 

0.3819 

.53768 

3 

.0906 

.0861 

1 

133 

0.6517 

.42094 

4 

.1190 

.1180 

1 

133 

0.1377 

.71121 

4 

.0861 

.0753 

1 

134 

1.5815 

.21072 

5 

.1353 

.1326 

1 

130 

0.4063 

32497 

5 

.1047 

.0998 

1 

131 

0.7245 

.39623 

6 

.1353 

.1212 

1 

130 

2.1113 

.14863 

6 

.1047 

.0801 

1 

131 

3.6001 

.05997 

Category  11: 

Job  Knowledge 

Category  12:  Attitude  Toward  Job  Performance 

1 

.1587 

.1317 

2 

131 

2.1006 

.12648 

1 

.0722 

.0415 

2 

131 

2.1732 

.11790 

2 

.1317 

.1291 

1 

133 

0.3894 

.53370 

2 

.0415 

.0395 

I 

133 

0.2729 

.60225 

3 

.1304 

.1301 

1 

133 

0.0507 

.82218 

3 

.0378 

.G327 

3 

133 

0.7052 

.40255 

4 

.1301 

.1275 

1 

134 

0.4004 

.52794 

4 

.0327 

.0309 

1 

134 

0.2569 

.61306 

5 

.1587 

.1546 

1 

131 

0.6274 

.42973 

5 

.0722 

.0719 

1 

131 

0.0500 

.82335 

6 

.1587 

:i5Q8 

1 

131 

1.2186 

.27165 

6 

.0722 

.0569 

1 

131 

2.1634 

.14373 

Category  13:  Overall  Performance 

1 

.1224 

.0934 

2 

130 

2.1471 

.12095 

2 

.0934 

.0910 

1 

132 

0.3*99 

.55518 

3 

.0995 

.0933 

1 

132 

0.9172 

.33996 

4 

.0933 

.0909 

1 

133 

0.3503 

.55493 

5 

.1224 

.1184 

1 

130 

0.5902 

.44375 

6 

.1224 

.0948 

1 

130 

4.0844 

.04534 
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